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ABSTRACT 
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include the Cost and Operational Effectiveness Analysis, the Modular Command and 
Control Evaluation System (MCES), the Mission Oriented Approach (MOA), the 


Headquarters Evaluation Analysis Tool, Evolutionary Upgrade Paths, and Functional 


Decomposition methods. Existing MOEs from past military C*l analyses and academia are 


identified, as well as potential methodologies for developing new MOEs, such as MCES 
and MOA. A new structural framework is developed using Marine Corps doctrine in order 
to categorize MOEs with potential applicability to Marine Corps C’I system analyses. The 
framework is used to compare and contrast identified MOEs, including performance 
ranking based on expert opinion. Conclusions discuss the utility of different evaluation 
techniques and estimated performance of identified MOEs. Additionally, suggestions for 


higher resolution sub-categories are presented. 
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EXECUTIVE SUMMARY 


The ongoing technical revolution spawned by increasingly powerful and inexpensive 
information technology has given rise to what has been described as a "revolution in 
military affairs". This revolution, coupled with jointness issues and decreasing service 
budgets has provided an impetus to use emerging technology as a force multiplier on 
future battlefields. 

The military is seeking to exploit technology in order to provide integrated, 
interoperable systems that meet the command and control requirements of services with 
inherently different missions. Command functions, and the flow of information on the 
battlefield falls within the realm of Command, Control, Communications, Computers, and 
Intelligence (C’l). Increasing demand for imagery, access to national intelligence 
databases, sensor management, and wide area networks are just a few of the challenges 
facing the services in general, and the Marine Corps specifically. Past shortcomings in C*I 
system procurement, however, coupled with the high cost of emerging technologies, have 
made upgrades to current C*I systems and new C’l acquisitions the subject of intense 
analysis. 

This paper presents the results of an extensive literature review and survey analysis 
conducted to identify, compare, and contrast various C’I evaluation techniques, and 
existing C*I measures of effectiveness (MOEs). Existing MOEs from past military 
analyses and academia are identified, as well as potential methodologies for developing 


new MOEs. A structural framework is developed using Marine Corps doctrine in order to 


Ix 


categorize MOEs with potential applicability to Marine Corps C*I system analyses. The 
framework is used to compare and contrast identified MOEs, including performance 
ranking based on expert opinion. 

A multitude of CI evaluation methodologies exist, each approaching the 
problem of comparative analysis in a slightly different manner. Available techniques 
include the Cost and Operational Effectiveness Analysis (COEA), the Modular Command 
and Control Evaluation System (MCES), the Mission Oriented Approach (MOA), the 
Headquarters Effectiveness Analysis Tool (HEAT), evolutionary upgrade paths, and 
functional decomposition methods. 

The COEA is well established as the tool of choice for Marine Corps C’*I system 
comparative analyses during the concept exploration phase of the acquisition process. 
The COEA framework, however, is broad enough to accept input from other, potentially 
more precise, evaluation techniques. The strength of the MCES lies in its initial focus on 
problem definition and the environment, rather than a premature focus on a quantitative 
model. MCES also enjoys a commonality of terms and structure which simplify its 
application. The MOA focuses on obtaining required input from political and military 
decisionmakers prior to defining an analytical question. HEAT has been successfully 
applied to military C*I systems, and represents a systematic methodology using 
pre-defined measures. Evolutionary upgrade path concepts help provide greater 
resolution when comparing alternate systems with similar capabilities. Functional 


decompositions use varied approaches which couch an analysis in terms of required 


system capabilities. Although any hybrid combination of methodologies would only be 
developed in the context of the analysis at hand, one such combination that suggests itself 
might be a COEA which employs the first four modules of MCES for problem definition, 
and evolutionary upgrade paths for distinguishing the best of a subset of "good" systems. 

The identification of realistic MOEs is a common thread which binds the various 
C‘I evaluation methodologies. Unique MOEs may be required for a specific analysis, and 
can be generated using MCES, MOA, evolutionary upgrade paths, or functional 
decompositions. Importantly, many useful MOEs already exist from past analyses, C*I 
literature, and academia. The ACCES, for instance, uses a pre-defined set of measures 
which remain constant from analysis to analysis. The proposed framework for 
categorizing MOEs in Chapter III, based on relevant doctrine, provides a tool for 
identifying and maintaining those MOEs with broad applicability to Marine Corps C’‘l 
system evaluation. The framework can change dynamically with emerging doctrine, and 
can also incorporate new MOEs developed by the application of MCES or other MOE 
generating methodologies. 

Importantly, the framework provides a starting point for increased resolution in 
categorizing C*I MOEs for use in Marine Corps related analysis. Potential sub-categories 
that may provide greater detail include functionality (i.e., Is a component primarily for C’, 
communications, intelligence, or interoperability?), and compatibility (i.e., Is the system 


employable inside existing vehicle spaces and command post configurations?). 


I. INTRODUCTION 


A. BACKGROUND 

The technological revolution spawned by the increasingly powerful and inexpensive 
micro-chip has given rise to what many call the "information age." Just as personal 
computers have transformed the commercial and private sectors, the marriage of 
computers and increasingly capable transmission systems, including satellites and 
microwave, have had a dramatic impact on the American military. Coupled with sweeping 
changes in technological capabilities are transformations caused by the cessation of the 
Cold War. 

Faced with ever decreasing budgets and end strength, the Army, Navy and Air Force 
are redefining service missions that once focused extensively on the former Soviet Union; 
the Marine Corps has enjoyed greater stability, since that service had only a limited role in 
the plan to defend Europe against the Warsaw Pact. Similarly, the issue of jointness has 
become dominant, meaning all programs, especially those dealing with information flow, 
are measured by their potential impact on the battlefield interaction of different services. 

Traditionally, the flow of information on the battlefield has been called "command and 
control," or C’?. For the purposes of this paper, command is a function of authority and 
responsibility assigned to an individual, while control is the means by which the 
commander exercises command. Taken together C* may be defined as the exercise of 
authority over designated forces in the accomplishment of a mission. C?’ has been 


described as the most important activity in war, although C’ "will not make a single attack 








against an enemy force, destroy a single target, or effect a single emergency resupply, .. . 
without C*, campaigns, battles and organized engagements are impossible." [Ref. 1]. As 
will be defined later, several terms exist to describe the equipment and procedures used for 
command and control, which will initially be described as command, control, 
communications, computers, and intelligence (C‘I) systems. The Department of Defense 
currently has in use, or under development, a multitude of C*I systems, all striving to meet 
the requirements implicit in the definition of command and control. 

The military is seeking to exploit rapid advances in C‘I technology, while 
simultaneously working for integrated, interoperable systems that meet the command and 
control requirements of inherently different service missions. DoD Direction, as stated in 
the Corporate Information Management initiative and "C4I for the Warrior” [Ref. 2] has 
outlined the need for more use of Commercial-off-the-shelf (COTS) items and looser 
business-driven standards [Ref. 3]. Increasing demand for imagery, access to national 
intelligence databases, sensor management, and wide area networks are just a few of the 
challenges facing the services in general, and the Marine Corps specifically. 

In the Marine Corps, current systems vary greatly in age and capability. New systems 
often represent quantum leaps in capability due to the age of the systems they replace, 
although the "new" system often represents less then cutting edge technology due to the 
length of the acquisition process. The new Single Channel Ground-Air Radio System 
(SINCGARS), which is 1980's technology, is currently being fielded to Fleet Marine Force 


units to replace a family of radios representing 1950's technology, which was self fielded 


in the 1960's. Additionally, most of the C*I systems currently in use were constructed as 
stand-alone systems designed to meet requirements at specific echelons of command or 
within certain warfighting areas (e.g., air defense). Such design has hindered attempts to 
modernize existing C*I systems, since seamless connectivity between echelons, warfighting 
areas, and services is necessary. For example, the digital "backbone" of today's Marine 
Corps employs approximately nine different pieces of communications equipment, and 
extends downward only to the regimental level [Ref. 4]. Even those newer equipments 
fielded for the joint arena are not immune to connectivity problems; the Tri-Service 
Tactical (Tri-Tac) Unit Level Circuit Switch (ULCS), an Air Force program jointly 
endorsed by the Army and Marines, has been purchased in quantity only by the Marine 
Corps, causing significant interoperability problems between Marine, Army and Air Force 
field headquarters [Ref. 5]. 

The effort to manage the controlled flow of huge quantities of information over vast 
distances, to increasingly smaller units, has resulted in tremendous challenges to the 
Marine Corps research and development (R&D) community. Due to the rapid changes in 
technology, and the large dollar amounts involved, upgrades to current systems and new 
system acquisitions are the focus of intense analysis. While analysis techniques vary, the 
requirement to identify appropriate measures of effectiveness (MOEs) is fundamental to 


each. An effective MOE provides the necessary framework to move an analysis from a 


qualitative to a quantitative assessment of the effectiveness of a C’I system. Although the 


definition will be expanded further, an MOE will initially be defined as a quantity that 





results from the comparison of various measures of a systems performance to the mission 
requirements [Ref 6]. If MOEs are found which can be applied across systems, 
comparisons between systems may be facilitated. Ultimately, the difficulties encountered 
by some C*I systems during concept development, acquisition, or fielding may be the 


direct result of less than robust MOEs. 


B. PROBLEM STATEMENT 

A wide variety of methodologies have been developed to evaluate C‘I systems, 
including the modular command and control structure (MCES), the mission oriv:ited 
approach (MOA), the cost and operational effectiveness analysis (COEA), and 
evolutionary upgrade paths. A common thread which binds these and other approaches is 
the fundamental requirement to identify robust, realistic, MOEs. This thesis identifies, 
documents, compares, and contrasts techniques for developing C*I MOEs, and identifies 
and examines existing MOEs, especially those with recurring use, that may be broadly 
applicable to existing Marine Corps C*I programs. The usefulness, definition, and 
rationale for each of the MOEs are presented, as well as the procedures for their 


calculation. 


C. APPROACH AND SCOPE 
A comprehensive literature search has been conducted in order to fully document 
existing methodologies for comparing C*I systems, focusing on techniques for developing 


C*I MOEs. Frequently used MOEs are identified, as well as MOEs which may have 


applicability across different C*I systems. The nature of the MOEs identified, as well as 
the procedures for their calculation, and the feasibility of the measure with regard to cost 
and time, are compared with other measures to determine contrasts. The study of MOEs 


focuses on those that may prove useful to the Marine Corps. 


D. OVERVIEW OF REMAINING CHAPTERS 

1. Chapter I. Literature Review 

This chapter presents a current overview of the large body of literature dealing with 
C’'l effectiveness measures. Key terms are defined, and a standard set of terminology is 
established. Each of the techniques for evaluating C*I systems is looked at in overview in 
order to provide a framework of how MOEs, once established, are employed. The 
techniques for MOE development inherent in each evaluation methodology are then 
presented, followed by a discussion of the general nature of the MOEs resulting from 
each. 

2. Chapter III. Analysis of Measures 

This chapter establishes a structural framework in which to place Marine Corps C? 
MOEs. The framework is derived from Fleet Marine Force Manual (FMFM) 3 
"Command and Control," and is inclusive of current Marine Corps C’ doctrine. Next, 
pertinent C? MOEs from the literature and past analyses are positioned inside the doctrinal 
framework, which consists of five category "bins," in order to facilitate analysis of the 
measures. Expert opinion is garnered to rank the MOEs on an interval scale within 


categories, according to the criteria of performance. Finally, in conjunction with 


Appendix A, a discussion of MOEs by category is presented, defining their individual 
characteristics, completeness, scalability, and underlying measures. Utilizing the 
underlying measures, the procedures for collecting measurable data and computational 
requirements are discussed. 

3. Chapter IV. Conclusions 

This chapter presents a summary of C’I evaluation methodologies and techniques for 
generating C‘I MOEs. The strengths of each evaluation method are reviewed, as well as 
potential hybrid combinations of methodologies. Finally, an overview of the doctrinal 
categorical framework is presented, along with suggestions for resolution enhancements to 


the existing categories. 


ll. LITERATURE REVIEW 


A. DEFINITIONS 

1. Why C*I? 

The continuing rapid rate of change in electronics and communications technologies 
has resulted in a serious lack of uniformity regarding terminology associated with 
command and control. Different military services, and indeed branches within services, 
lack agreement in the definitional framework of C? [Ref. 7]. In the years following World 
War II, the familiar term command and contro] was expanded to include communications, 
becoming C’. Intelligence was added to create the C’l definition, which was standard for 
almost thirty years [Ref. 8]. This paper will use the term command, control, 
communications, computers and intelligence (C*I), which reflects current usage in the 
Marine Corps (e.g., Assistant Chief of Staff, C*l), and which participants of the 1993 
Command and Control Concept Development Game agreed should become standard for 
the naval service, in order "to avoid unnecessary confusion among ourselves and with 
other services and remove any differences between the naval service's approach to C*I and 
the Joint Staffs." [Ref. 8] 

The breadth of the C? domain, however, continues to give rise to a multitude of 
similar terms. The increased reliance on large intelligence databases and service 
interoperability requirements has given rise to "command, control, communications, 


computers intelligence and interoperability (C‘I’)," and the electronic warfare community 


frequently adds EW to the above terms, as illustrated by the 1992 Military Operations 
Research Society (MORS) C*IEW Measures of Effectiveness Conference. 

A starting point for the definition of command and control was defined in Chapter I 
as the exercise of authority over designated forces in the accomplishment of a mission. 
More specifically, Joint Chiefs of Staff Publication I (JCS Pub 1) defines Command and 


Control as 


The exercise of authority and direction by a properly designated 
commander over assigned forces in the accomplishment of his mission. 
Command and control functions are performed through an arrangement of 
personnel, equipment, communications, facilities, and procedures which are 
employed by a commander in planning, directing coordinating, and controlling 
forces and operations in the accomplishment of this mission. 


Thus command and control is by definition composed of three major elements: 
personnel, equipment, and procedures. In keeping with the intent of past MORS, NPS, 
and MITRE workshops dealing with the subject of C? analysis, C? should be taken to mean 
as broad a concept as possible, avoiding any distinction between, for example, "command 
and control" and "combat direction." [Ref. 7] Since this paper deals with the identification 
of MOEs to aid in the quantifiable analysis of C*I "systems," the concept of a C*l system 
must be defined. 

Again turning to JCS Pub 1, a C? system is defined as "the facilities, equipment, 
communications, procedures, and personnel essential to a commander for planning, 
directing, and controlling operations.". The Marine Corps has described a C*I system as 


"the integration of communications, computers and intelligence technologies and 


procedures into a functional, cohesive system designed to support the commander . . . C’I 
is distinct from C’ in that C’l is the enabler for C*." [Ref. 8] Although emerging Marine 
Corps doctrine seems to define the components of a C’ system as people, information, and 
C’ support structure, for purposes of this paper a C*I system will be viewed as having as 
its three components: physical entities, structures, and processes [Ref. 1]. Again using 
definitions from MORS workshops and the MCES system, a C’I system incorporates the 
three elements of the JCS Pub 1 definition in the following manner. 

a. Physical Entities 

Physical entities refers to the equipment, software, facilities, and manpower of a 
C’I system. Manpower, or people, "drive the command and control system — they make 
things happen — and the rest of the system exists only to serve them." [Ref. 1] In other 
words, humans are integral components of the C*l system, not merely users [Ref. 1]. 
Facilities refer to "the command posts and other spaces where command and control is 
sometimes performed," while equipment includes "the communications and other gear 
which support command and control, from pyrotechnics to computers." [Ref. 1] 

b. Structures 

Structures of a C’I system identify the arrangement and interrelationships of 
physical entities, procedures, protocols, concepts of operation, and information patterns 
[Ref. 9]. Such arrangements can be both spatial and temporal. Procedures include the 


"techniques by which we perform various command and control functions," and they apply 


only to "rote or mechanical tasks, not to acts of judgement." [Ref. 1] Organization 





"provides those groups of people through which the commander exercises command . . . 
and establishes the channels by which information flows." [Ref. 1] 
c. Processes 
A process is a description of what the system is actually doing, and reflects the 
functions carried out by the system [Ref. 1]. These activities "include — but are not limited 
to — gathering and analyzing information, making decisions, organizing resources, 
planning, communicating instructions and other information, coordinating, monitoring 
results, and supervising execution." [Ref. 1] Note that processes are not the same as 
procedures. Both processes and functions are defined in detail in the Marine Corps 
Mission Area Analysis of Mission Area 1] —Command and Control. 
2. Measures 
Within the realm of C’I system evaluation, the set of terms that has been developed 
by the analytic community seems to enjoy a more uniform acceptance than the previous 
terms dealing with the broader descriptions of C>. Measures of CI systems cover a 
spectrum of specific system aspects. The phrase "measures of merit" (MOM) has been 
used to describe the overall collection of measures, which will be categorized according to 
four groups: measures of force effectiveness (MOFEs), measures of effectiveness 
(MOEs), measures of performance (MOPs), and physical (sometimes called dimensional) 
parameters [Ref. 9]. A hierarchical relationship exists among MOMs, as portrayed in 


Figure }. 
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Figure 1. Hierarchical Relationship of Measures of Merit. From Ref. [9]. 


MOMs, as listed, are categorized by their relationship to the boundaries of a C‘I 
system. In turn, the boundary is defined by the scope of the analysis being conducted, and 
represents the delineation between the system being studied and the environment. 
Therefore the definitions of the four categories of measures, while rigorous, can only be 
applied in their relationship to the system boundary. Figure 2 depicts the relationship 
between different boundaries and the environment, in what is called the "onion skin" 


diagram. Thus a thorough understanding of the boundaries for any system must be 


accomplished within the context of the analysis at hand. Depending on the resolution of 





the analysis, what might be used as an MOP (for a national level system), could be used 


in another analysis as an MOE (for a service level system). [Ref. 7] 
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Figure 2. Onion Skin Diagram. From Reef. [7]. 
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Besides category and system reference (or boundary) attributes, all MOMs have the 
additional attributes of names, function reference (or purpose), units of measure, value 
measured, and threshold value (or goal) [Ref. 9] 

a. Physical Parameters 

A physical parameter describes the tangible, physical aspects of a component or 
system. Its value determines system behavior and the structure under question even when 
at rest [Ref. 7]. Examples of physical parameters include size, weight, capacity, and 
number of pixels. 

b. Measures of Performance 

An MOP is closely related to a physical parameter, in that it represents a 
property or characteristic inherent in the physical entity [Ref. 9]. Stated in another way, it 
measures attributes of systems behavior, or "performance is a measure of the ability to 
take action in accordance with requirements in the execution of an action.” [Ref. 6] 
Examples include baud rate, throughput, and frequency range. MOPs are internal to the 
system being analyzed, and are scenario independent [Ref. 9]. 

MOPs have also been defined as non-probablistic measures of performance, 
where "the MOP class provides for the collection of metrics . . . that are not probabilities 
of successful outcomes of functions." [Ref. 10] Thus MOPs are the "consequence" of 


specific configurations of physical elements. The probabilistic nature of this definition of 


MOPs also leads to an alternate definition of MOEs. 





c. Measures of Effectiveness 

An MOE is a measure of how the C’I system performs its functions, or affects 
other entities, within an operational environment [Ref. 9]. C*I MOE are measured 
relative to some standard, which is often simply the implicit baseline of "how would a 
perfect C*I system perform." [Ref. 9] Additionally, C*] MOE are scenario dependent. 
Some examples include probability of detection and target detection; Chapter III will 
provide many more examples in the form of CI MOEs with potential applicability to 
Marine Corps systems. 

By couching the definition in probabilistic terms, Girard [Ref. 10] and Elele [Ref. 
6] have developed more mathematically rigorous definitions for MOEs. In Girard's terms, 
an MOE is the probability of the successful accomplishment of a function, where all 
probabilities are conditional, and are derived from MOPs and lower level (or prior) MOEs, 
and where a function is a process relating in an outcome [Ref. 10]. Thus “an MOE 
defined by an objective function at an upper level is a dependent variable, and is a 
mathematical function of the MOEs defined by objective functions at a lower level." [Ref. 
10] Ultimately, an "audit trail" equation is generated, linking the conditional upper level 
MOE to measurable MOPs. Elele uses Bayes' Rule to develop a similar probability based 
MOE definition [Ref. 6]. 

At an even higher level of abstraction, Levis and Cothier [Ref. 11], Levis [Ref. 
12], and Levis and Martin [Ref. 13] have developed an MOE methodology based on set 


theory. Similarly, Dockery [Ref. 14] explores fuzzy set theory in defining MOEs. 
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d. Measures of Force Effectiveness 
An MOFE is a measure of how the force performs its missions, or its 
"contribution to battle outcome." [Ref. 7] | Again, MOFEs are inherently scenario 


dependent, since they are tied to operational context and assumed enemy actions. 


B. C‘I SYSTEM EVALUATION TECHNIQUES 


1. Cost and Operational Effectiveness Analysis (COEA) 


The MOEs and doctrinal framework presented later in chapter III will be developed 


primarily for use with the cost and operational effectiveness analysis. A COEA is a 
systematic study "designed to assist a decision maker in identifying a preferred choice 
among possible alternatives." [Ref. 15] COEAs are currently the tool of choice for the 
evaluation of alternative systems during the concept development phase of acquisition. 

The first step in a COEA involves defining the overall objectives of the study, 
identifying a "base case” system (usually consisting of currently fielded assets), and a set of 
alternative systems to be evaluated. The evaluation itself usually focuses on three areas: 
effectiveness, cost, and risk, of which effectiveness requires the development of MOEs 
that can be applied equally to the base case and each alternative. Figure 3 shows a normal 


COEA methodology. 
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Figure 3. Cost And Operational Effectiveness Analysis. From Ref. [15]. 




















2. Modular Command and Control Evaluation System (MCES) 

The modular command and control evaluation system was developed by a team of 
experts from industry, government and academia, in a series of forums initiated by the 
MITRE Corporation and the Military Operations Research Society, and hosted by the 
Naval Postgraduate School during the 1980s. The purpose of the forums was to develop 
a general approach to analyzing C* systems, and the result of their efforts is a seven 


module system that has been successfully applied to many facets of C’, including planning, 
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acquisition, testing and operation [Ref. 16]. The strength of the MCES approach lies in 
the guidance it provides to analysts who might otherwise focus prematurely on the 
quantitative model, rather than the problem definition and the specific measures needed to 
discriminate between alternatives [Ref. 16]. The MCES presents a method to attack 
difficult concepts in a standardized manner, using some semblance of standardized 
terminology and paradigms. Ultimately the MCES can be thought of as two processes, a 
managerial system which serves as a guide to specifying the problem to be analyzed, and 
an analytic system which serves as a guide to the analysis process itself [Ref. 17]. The 
seven modules are described individually below. 

The first module, problem formulation, identifies the objective of an application, 
leading to a precise statement of the problem being addressed. The system's operational 
and deployment concepts, environmental factors, scenarios, assumptions and threats are 
made explicit. [Ref. 17] 

Module two, system bounding, takes the output of the first module (the problem 
statement), and uses it to bound the system. The system bounds are defined in terms of 
the three categories of a C*I system (defined earlier in the chapter as physical entities, 
structures, and processes), and the onion skin diagram. The implementation of this 
module results in the identification and categorization of the system elements. [Ref. 17] 

Module three, process definition, involves building a dynamic framework that 
identifies the relevant C’ processes, each broken down into a set of functions. The 


framework focuses attention on the environmental forces which initiate the C’ process, the 


internal C* process functions which detail what the system is doing, and the required input 
to and output from the internal C? process. The implementation of this module results in 
the identification of the functions of the C? process, which are mapped into a C’ process 
loop. The generic C? process loop consists of an environmental stimulus, which initiates a 
process of: sense, assess (with regard to desired state), generate, select, plan, and direct, 
the result of which is a system response to one's own forces, inside the context of the 
environment. [Ref. 17] 

The fourth module of the process, integration of system elements and functions, 
integrates the system elements identified in module two, and the process functions from 
module three, into a representation of the C*I system. The product of this module is at 
least a complete descriptive conceptual model, and perhaps a complete mathematical 
model. [Ref. 17] 

Using the four prior modules as a base, the fifth module, specification of measures, 
specifically identifies the MOPs, MOEs, and MOFEs necessary to address the problem of 
interest. The measures are specific with regard to the corresponding levels of the C’l 
system and C‘I functions. Resulting measures must adhere to a set of desired 
characteristics, which often includes whether the measure is mission-oriented, 
discriminatory, measurable, objective, sensitive, and inclusive [Ref. 17]. Note that the 
MCES can be utilized just for the implementation of the first five modules, which will 


result in a concise problem statement, a complete description of the C’I system, including 
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its functions, and a list of acceptable measures. The set of measures can then be applied in 
the form of another C’ evaluation methodology, such as a COEA. 

Module six, data generation, involves the generation of data for the measures 
identified in the previous module. Any appropriate type of data generator may be used, 
including simulation, exercises, experiments, or expert opinion [Ref. 17]. The module 
output is simply the values associated with the measures of interest. 

The final module, aggregation of measures, aggregates and interprets the observed 
values of the measures. The analysis results are tailored to address the problem statement 
of module one. The seven MCES modules are performed iteratively, with the decision 


maker located as shown in Figure 4. [Ref. 17] 
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Figure 4. MCES Structure. From Ref. [17]. 
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3. Mission Oriented Approach 

The mission oriented approach is a four phase framework for formulating 
requirements while maintaining an optimal balance between operational requirements, 
technical capabilities, and scarce resources (see Figure 5) [Ref. 18]. The MOA 
"systematically relates mission objectives and operational plans to C’ system objectives 
and associated programs." [Ref. 18] The approach is somewhat similar to the MCES, in 
that the first two phases of MOA relate to the functional definition module of MCES, and 
the last two phases of MOA deal with the C? capabilities in MCES module four. Like the 
MCES, the MOA is beneficial as a potentially common framework for discussions among 
communities, although it lacks the commonalty of terms built into MCES. Finally, while 
the MOA is well suited for requirements definition, it is not particularly well suited for 
evaluation of alternative C’I systems [Ref. 19] 

The first phase begins by addressing the question "What are we trying to achieve 
operationally?" This question is directed well above the operator and analyst level, to the 
relevant political and military decision makers. The response to this question is generally 
framed in terms of a set of strategic capability objectives for employing forces. These 


force capability objectives become the standard against which the capabilities of existing, 


and proposed, "packages" of C? systems can be measured. [Ref. 18] 
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Figure 5. Generic MOA. From Ref. [18]. 
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Phase two addresses the related question, "How are we trying to perform the 
mission?" This question should be answered by those operational personnel who 
formulate concepts of operation at multiple levels; typical levels of aggregation include 
strategic capability, mission (operational) capability, and functional capability. For each 
level, the capability objectives are "self-consistently derived, beginning with specified 
strategic capability levels, based on perceived adversary operations, friendly concepts of 
operation and environmental factors." [Ref. 18] 

The third phase of the MOA, addresses "what technically" is required to support the 
operation defined in the first two phases. Technical personnel are employed to "translate 
the operational/mission/function capability objective levels into the technical attributes of 
the packages of communications" needed to implement the respective capability levels 
[Ref. 18]. Technical characteristics are derived using the existing, and proposed, 
characteristics of friendly forces, enemy forces and the environment. 

Having defined "what technically" is required in the third phase, the fourth phase of 


the process describes "how technically" will the desired characteristics be incorporated. 


Based on technical deviations obtained by comparing existing packages of C’ systems to 


the technical objectives of phase three, alternative packages of C’ systems can be 
formulated "consistent with assumed restrictions on available resources.” [Ref. 18] Thus 
potential "investment strategies” are outlined, which can then be assessed to determine 
how well they meet the strategic objectives set forth by policy makers in phase one [Ref. 


18]. Note that by going back to phase one, the MOA becomes an iterative process, 





indeed "the key to successful application of MOA is iteration." [Ref. 18] The iteration is 
needed to address issues across communities (e.g., operational and technical), and within 
communities (e.g., system operators and maintainers). 

4. Headquarters Effectiveness Analysis Tool 

The headquarters effectiveness evaluation tool, or HEAT, is based on adaptive 
control system theory [Ref. 20]. Two variants of HEAT are currently in use, they are the 
Army command and control evaluation system (ACCES), and a Navy version for battle 
force in-port training (BFIT) [Ref. 21]. The primary focus of HEAT is the overall 
performance of a headquarters, and in general HEAT addresses the question "What should 
be done to improve the effectiveness of a military headquarters?" [Ref. 21] ACCES 
focuses this question primarily on the division headquarters [Ref. 20]. Candidates for 
improvement include the headquarters staff (staff size, organization, training, procedures, 
etc.), and C*I systems. In execution, HEAT uses specific definitions of C’I theory, tools 
and applications. 

Theory, in the form of a decision-making model, aids in the design of measurement 
tools, and suggests hypotheses for experiments and exercises. The decision making model 
is "the most fundamental concept in HEAT theory." [Ref. 21] The model itself is an 
expanded Boyd cycle, and includes the six "process steps" of: monitoring (obtaining data 
from the environment), estimation (placing values on monitoring), situation assessment 
(interpreting estimates), option generation (generates feasible courses of action), option 


selection, and plan-generation/direction [Ref. 21]. The model "asserts that planning, 
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decisionmaking, and execution activities conducted by a headquarters are analogous to a 
feedback control system that attempts to keep its external environment within acceptable 
bounds, despite the actions of the enemy." [Ref. 21]. The six process steps transcend 
organizational boundaries, since each step does exist (in some form) in virtually every 
headquarters, making the model a foundation for posing and answering questions about 
headquarters effectiveness. 

The HEAT measurement tool consists of a set of measures that, as a whole, capture 
important determinants of effectiveness. ACCES in particular, has forty-three MOEs 
designed to determine command post effectiveness [Ref. 20]. The HEAT measures fall 
into three broad categories: process, performance, and effectiveness. Process and 
performance measures can be further defined under four types: characteristic measures 
(process measures that indicate whether a headquarters strives to be optimal), 
coordination measures (process measures that capture the degree of information 
coherence), queueing measures (performance measures that indicate the origins of time 
delays), and quality measures (performance measures that indicate the degree to which 


performance approaches a norm — usually the accuracy of the headquarters perception of 


ground truth) [Ref. 21]. HEAT MOEs that may be pertinent to Marine C*I systems are 


included in Chapter IJ of this paper. 
A HEAT application results in the assignment of values to the HEAT measures, and 
may consist of experiments and/or exercises. Note that experiments and exercises are 


complementary, as exercises can help validate experiment findings, or suggest independent 





variables for future experiments. A key aspect of a HEAT application is the requirement 
for observation. Towards this end, HEAT observers must ‘undergo specialized training, 
which includes fundamental HEAT concepts and the rudimentaries of data collection [Ref. 
21]. 

Although the HEAT methodology is well defined, and has been successfully applied 
by analysts, the reliance on a defined set of MOEs limits this evaluation technique when 
compared to the MCES or the MOA. Additionally, HEAT uses a standardized model, in 
contrast to the formal problem and environmental definitions found in the initial modules 
of the MCES. The Army has found that "the most debilitating problem with the ACCES 
methodology is the reliance upon percentage figures ... raw scores would provide a great 
deal more precise and reliable information." [Ref. 20] 

5. Evolutionary Upgrade Paths 

Most existing C*I systems, especially recent procurements, will likely be 
incorporated with some type of product improvement or upgrade during their useful life 
cycle. The concept of evolutionary upgrade paths is that an effective evaluation of a new 
C’I system must include an evaluation of its planned upgrade path toward some goal or 
target level of functionality. The evolutionary acquisition concept is a "build a little, test a 
little, field a little" approach, using off-the-shelf equipment and software where applicable. 
More formally, it is "an acquisition strategy which may be used to procure a system 
(which is) expected to evolve during development, within an approved architectural 


framework, to achieve an overall systems capability." [Ref. 19] 
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Proponents of evolutionary acquisition point to several compelling reasons for 
utilizing the approach. Past attempts at using the "big system" approach, that is, fielding a 
complete, integrated system all at once, have been somewhat unsuccessful. Examples of 
perceived failures include the many iterations of TCO (starting in 1968), and MIFASS 
[Ref. 19]. Additionally, it is difficult to generate a complete list of defined requirements 
for a needed system, new technology, new procedures, and the length of the acquisition 
process all add to the uncertainty of predicting the final system requirements. Finally, 
proponents argue that user response is more quickly incorporated and capabilities are 
fielded faster using evolutionary acquisition. By starting with a fielded system with which 
the user is already familiar, the user can better assimilate and evaluate changes. Also, by 
quickly fielding a core capability while concurrently developing component systems, new 
technology (including commercial-off-the-shelf items) can reach the user at a rate much 
faster than is currently possible. 

The evolutionary upgrade framework consists of four major steps. Step one is to 
define the target system's functions and capabilities. The system capability requirements 
are detailed in the originating mission need statement, and the operational requirements 
document. Similarly, C*I functions are well documented in doctrine, as well as the 
previously mentioned Marine Corps Mission Area 11 analysis, although functional 
decompositions may be required to meet the desired resolution. Once defined, functions 


must be matched to the technological capabilities required to support them. With the 
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relationship between target system capabilities and functions established, weights are 
assigned to the function according to perceived value. 

The second step is to define all viable upgrade paths. For each potential base 
system, the goal is to establish a list of candidate paths which result in a target system that 
possesses the capabilities established in the original definition of the target system. Base 
systems should preferably meet some of the requirements of the target system, and may 
have multiple candidate paths. The underlying concept is that each base system could 
someday meet the total requirements of the target system, by incrementally adding future 
capabilities. Finally, each candidate path should satisfy the current cost, resource, and risk 
constraints. 

The third step of the framework involves determining the overall value and cost of 
each candidate path. The values for each function were determined in step one, and by 
summing the values of the functions met while incorporating some "discount" function to 
account for the timeliness with which the base system reaches target system maturity. The 
time series approach to discounting is necessary since, by definition, each candidate path 
ultimately incorporates each of the functions required in the target system, which would 
cause the sum of the function values for each path to be equal. Additionally, the concept 
of "functionally derived values," or attaching greater values to the most important 
technological capabilities, can provide valuable insights into system priorities. [Ref. 19] 

The cost of each candidate path can be computed by adding the cost of the 


alternative base system if bought today, and the cost of adding a new capability in the 
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future. Note that the costs to achieve the target system may differ between base systems, 
due to the potential differences in hardware, software, and configurations. Costs should 
be put in constant dollars, and should include costs associated with research, development, 
testing, and evaluation (RDT&E), procurement, and operations and support costs. Like 
values, costs can be "discounted" to account for technological priorities. [Ref. 19] 

The final step can be thought of as a linear program, where one is maximizing value, 
subject to cost, resource, and risk constraints. The final selection method would involve 
the elimination of those candidate paths that violated the constraints, and selecting from 
the remaining paths the one with the highest value. 

6. Functional Decompositions 

To an extent, some form of functional decomposition is present in most of the 
preceding techniques, such as the detailed user requirements in the COEA, module three 
in the MCES, or "how operationally" in the MOA. Several less used C‘I evaluation 
techniques employ variations of functional decomposition as the primary framework for 
their methodology. 

Kemple, Stephens, and Crolotte [Ref. 22] use expert opinion in a three part 
methodology that defines MOEs for CI systems. First, a three phase "value analysis” is 
conducted where experts define "attributes" of system effectiveness, weight them by 
relative importance, and then reduce them to a reasonably sized set of the most significant 
attributes. Next a functional analysis is conducted, using a system functional 


decomposition which is again weighted by experts, in order to facilitate comparisons. 
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Finally, the value and functional analyses are combined to generate what should be 
measured, appropriate scenarios, and MOEs defined through linear aggregation. [Ref. 22] 
Girard [Ref. 10], has developed what he calls "a function based definition of C? 
MOEs." Using requirements documents, such as the Navy's Top Level Warfare 
Requirement (TLWR), a functional decomposition is conducted in each of three domains: 
mission, organization, and resource. Mission success criteria (MSCs) and required 
capabilities (RCs), both established by the TLWR, are used to generate the initial tiers of 
functions. Most mission success criteria and required capabilities can be stated in terms of 
the probability of successfully accomplishing a function, therefore a function defines its 
own metric. Conversely, a probability metric defines the outcome of its related function. 
For example, the probability of detection identifies the detection event as the outcome of 
the search function; functions, and their related metrics, become inseparable. [Ref. 10] 
RCs can be thought of as a set of subfunctions for the higher level functions defined 
by the MSCs. In this way, a procedure (a set of functions carried out in some prescribed 
manner) represents the implementation of the higher order function in terms of its 
subfunctions. A model of the procedure will be nothing more than a conditional 
probability describing the relationship of the lower level metrics to the higher level metrics. 
In this manner, Girard arrives at his definition of an MOE as the (conditional) probability 
of the successful accomplishment of a function [Ref. 10]. The MOEs resulting from the 


decomposition can then be applied using a variety of analytical tools. 
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The C’ MOE Handbook issued by the Army's Training and Doctrine Command 
(TRADOC) also uses functional decomposition methodology to describe a method for 
developing MOEs. As a first step, the methodology lists six components which must be 
defined: objectives, functions, processes, tasks, structures, and physical entities. These 
components represent "links" between the C*I system, the decisionmaker's questions, and 
the hierarchy of C*l components. A rough outline of the links can help portray 
interrelationships and implications inherent in the decisionmaker's questions. The 
questions themselves should be simplified, with the goal being "to break those high level 
questions down through a systematic functional decomposition that results in the 
development of issues or questions that are specifically worded to obtain data." [Ref. 23]. 
Through the functional decomposition of the decisionmaker's questions, issues, sub-issues, 
essential elements of analysis, MOFEs. MOEs MOPs, and data requirements are 
developed which guide and direct the required analytical work. The handbook 
recommends a logic tree (dendritic network) to portray the interconnectivity of a logical 


network used in decomposing the decisionmaker's questions into issues. [Ref. 23] 


C. SUMMARY 

Although the COEA is the standard tool employed by the Marine Corps to evaluate 
alternative C“I systems during the concept definition phase of acquisition, this chapter has 
suggested that there are several equally robust evaluation tools. Significantly, alternate 
evaluation techniques can be complementary to the traditional COEA framework. An 


application of one of the functional decomposition techniques can strengthen the 


31 


originating requirements documents, the MNS and the ORD. The MCES, with its 
commonalty of terms and robust problem definition, can aid in the identification of 
measures that will provide a valid comparison between the alternative systems in the 
COEA. Finally, the evolutionary upgrade path framework can add resolution to 
alternative systems that in the current COEA framework might otherwise "be too close to 


call." 
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II. IDENTIFICATION AND ANALYSIS OF MEASURES 


A. BACKGROUND 

Due to the variety of functions different C*l systems may perform, the MOEs to be 
used in an analysis must often be devised specifically for the task at hand. Indeed, the 
MCES methodology makes clear that prior to identifying MOEs, the problem must be 
clearly stated and the environment must be fully defined. As has already been mentioned, 
an MOE at one level might be an MOFE or an MOP at another level. 

Many good MOEs, however, are already in existence, and there is no requirement to 
be original in identifying MOEs for a particular study. Pertinent MOEs may be gleaned 
from past analyses, studies, and academic submissions to those periodicals that deal with 
command and control (e.g., Signal Magazine and IEEE). When dealing with the large 
body of existing MOEs, the primary problem is determining which are applicable and 
valid. Further, the sheer volume of existing MOEs requires some sort of framework to 
categorize them; a workable framework allows manageable comparisons to be made 
between potential alternatives. Several potential frameworks present themselves for 
consideration, including categorizing by system composition (hardware, software, 
communications equipment), system functions, or system cost. A system based 
categorization, however, fails to narrow the set of existing MOEs with regard to Marine 
Corps C'l directives. To that end, existing USMC doctrine pertinent to the C’I arena was 
reviewed in order to gain insight into a framework with more specific application to the 


Marine Corps. This framework will be summarized in the sections that follow. 
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B. DOCTRINAL FRAMEWORK 

Marine Corps doctrine concerning command and control, while voluminous, is also 
very specific in regard to required capabilities and characteristics. FMFM 3, Command 
and Control, provides a structural framework for the classification of C’ system ideas, 
requirements, capabilities and characteristics listed in other doctrine, including MFM 
3-30 Communications, Mission Area Analysis of Mission Area 11 — Command and 
Control, FMFRP 14-30 A Concept of Command and Control, and The Command and 
Control Concept Final Report. FMFM 3 is particularly pertinent because it marks "the 
first (Marine Corps) effort to apply command and control theory specifically to maneuver 
warfare in doctrinal terms." [Ref. 24] Specifically, the framework suggested by FMFM 3 
consists of five broad categories, which seem to be inclusive of the crucial C’I system 
requirements discussed throughout Marine Corps doctrine. The doctrinal categories, 
taken directly from FMFM 3, are as follows. 

1) Facilitate Commander's Influence of Events 

2) Adapt to the Situation 

3) Support Information Requirements 

4) Exploit MAGTF Capabilities 

5) Support CP Effectiveness 
Broader descriptions of these categories are given in the sections that follow. 

In order to assess the extent to which these doctrinal categories encompass all Marine 


Corps C’*I doctrinal requirements, and to describe the doctrinal categories in greater detail, 
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additional Marine Corps C’I publications were examined for further elements of doctrine. 
Any statement that defines additional C*I performance or capability requirements was 
listed as an element of doctrine. Table 1 shows examples of these additional elements of 


doctrine, which total 39 statements from six publications. 





A C’ system "places decision making at the lowest possible level.” 





A C* system "ensures subordinate commanders receive ideas and concepts, not just data packages." 








A C’ system should "translate the commander's decisions into plans and orders." 





A C’ system should "supervise the execution of those plans and orders." 





"The MAGTF commander must be able to make and implement decisions faster than the enemy, so as to 


gain the initiative and create tactical advantages for the MAGTF to exploit." 





A C? system should possess the "capability for MEF and MEB command elements to conduct rapid 


mission planning at a level comparable to that of a MEU." 











Table 1. Examples of Additional Elements of Doctrine. 


The additional elements of doctrine, hereafter referred to as "supporting elements," 
were placed into the context of the five doctrinal categories in order to determine the 
comprehensiveness of the framework. The actual placement of supporting elements into 
the doctrinal categories was based on the author's determination of whether the supporting 
elements reinforced or clarified the description of a particular category. For example, the 
definition of doctrinal category one, "Facilitate Commander's Influence of Events," is as 


follows. 


No coherent battle plan is possible without the commander's vision of 
how he intends to win. The C’ system, imbued by his intent, provides the 
means to apply his decision effectively, gather and analyze information, 
implement his plan, and supervise his forces [Ref. 24]. 
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Thus the supporting element "translate the commander's decision into plans and orders” 
reinforces that portion of the doctrinal category definition which discusses the need for a 
"means to apply his decision effectively." Similarly, the supporting element which states 
that a C*I system should provide "the capability to view friendly and enemy force data as it 
is received" clarifies (and provides resolution to) the category definition assertion that a 
C’l system should help a commander "supervise his forces." 

In this manner, a subset of the 29 supporting elements were determined to fit within 
the first doctrinal category; Table 2 reflects this subset. The same logic was applied to fit 
supporting elements within the remaining four doctrinal categories, with some supporting 
elements appearing in more than one doctrinal category. Ultimately, all 39 supporting 
element statements were successfully placed into the doctrinal framework, suggesting that 
the framework is indeed inclusive of all Marine Corps C‘I doctrine. A summary of the 
distribution of supporting elements under each doctrinal category is included in the 
sections that follow. 

Once the inclusiveness of the doctrinal categories was established, MOEs selected 
from the literature were mapped into the doctrinal categories of the framework, and 
further related to the supporting elements already listed under each doctrinal category; 
the supporting elements were used to categorize the MOEs because they provided 
increased resolution to the doctrinal category definitions. The increased resolution, in 
turn, was deemed necessary due to the specificity of many of the MOEs. Like supporting 


elements, MOEs were put into the framework based on the author's interpretation of their 
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relationship to the categories. This interpretation was based on the stated definition of 


each MOE, as well as their respective statements of purpose. 








A C’ system "places decision making at the lowest possible level." 





A C’ system "ensures subordinate commanders receive ideas and concepts, not just data packages. 





AC? system should "translate the commander's decisions into plans and orders.” 








+ 


Ac system "supervise the execution of those plans and orders." 
= 


“The MAGTF commander must be able to make and implement decisions faster than the enemy, so as 





to gain the initiative and create tactical advantages for the MAGTF.” 





A C? system should possess the "capability for MEF and MEB command elements to conduct rapid 


mission planning at a level comparable to that of a MEU." 





|Ac? system should possess the "capability to view friendly and enemy force data as it is received.” 








AC? system should possess the "capability to complement the commander's situational awareness and 


instincts." 








"Support planning for future operations by offering the commander the ability to gain or create an image 
of the battlefield at ranges in time and distance beyond the scope of current operations.” 











Table 2. Subset of Supporting Elements for First Doctrinal Category. 


Sections 1 through 5 below present the details of the framework, with each section 
titled by the doctrinal category to be presented. Following the title is the definition of the 
doctrinal category. This definition is itself followed by the header "supporting elements,” 
under which are listed the supporting elements included in the doctrinal category. Each 
supporting element is preceded by a lower case letter (in alphabetical order, starting with 
(a)), which will be used to reference the respective supporting element in Tables 3 through 


Vs 
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The table which follows the list of supporting elements in each section portrays the 
MOEs included in the doctrinal category. Next to each MOE, an "X" is recorded below 
the letter of each supporting element that it measures. For example, in Table 3, the MOE 
“Number of Orders Issued" can be used to measure supporting elements (a) ("places 
decision making at the lowest possible level"), and (b) ("ensures subordinate commanders 
receive ideas and concepts, not just data packages"). An empirical indication of MOE 
performance within a doctrinal category can be determined by counting the number of 
supporting elements it measures. A full description of each MOE, including 
computational methods and references, is included in Appendix A. 

As part of this research, the MOEs within each doctrinal category are ranked 
according to the extent to which each provides a realistic assessment of how well a C’l 
system meets the stated C*I system requirements within the doctrinal category. The 
rankings for the MOEs within each doctrinal category are included in the last column of 
each table. Rankings were accomplished using expert opinion garnered by a 
questionnaire. Questionnaires were completed by twelve respondents, including eleven 
Marine Officers and one NPS Operations Research Department professor. The ranked 
lists were analyzed using a variant of the Method of Successive Interval Scaling, where 
experts are asked to rank instances in terms of how much of a property they possess [Ref. 


25]. The questionnaire and resulting calculations are included in Appendix B. 
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1. Doctrinal Category: Facilitate Commander's Influence of Events 

"No coherent battle plan is possible without the commander's vision of how he 
intends to win. The C’ system, imbued by his intent, provides the means to apply his 
decision effectively, gather and analyze information, implement his plan, and supervise his 
forces." [Ref. 24] 

a. Supporting Elements 

(a) A C’ system "places decision making at the lowest possible level." [Ref. 8] 

(b) A C’ system “ensures subordinate commanders receive ideas and concepts, 
not just data packages." [Ref. 8] 

(c) A C’ system should "translate the commander's decisions into plans and 
orders." [Ref. 26] 

(d) A C’ system should "supervise the execution of those plans and orders." 
[Ref. 26] 

(e) "The MAGTF commander must be able to make and implement decisions 
faster than the enemy, so as to gain the initiative and create tactical advantages for the 
MAGTF to exploit." [Ref. 1] 

(f) Should possess the "capability for MEF and MEB command elements to 
conduct rapid mission planning at a level comparable to that of a MEU." [Ref. 1] 

(g) Should possess the "capability to view friendly and enemy force data as it is 


received.” [Ref. 1] 
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(h) Should possess the "capability to complement the commander's situational 
awareness and instincts." [Ref. 1] 

(i) "Support planning for future operations by offering the commander the 
ability to gain or create an image of the battlefield at ranges in time and distance beyond 


the scope of current operations." [Ref. 1] 



















Measure of Effectiveness Measures These Supporting Performance 


Elements 


(c) | (d) | (@) | (f) | (g) | (h) | (i) 





System contribution to the commanders 
perception of the enemy. 








Percent action initiated by time ordered. 








Mean Dissemination Time. 


Percent orders clarification requested. X |X Xx 
x x 


Percent planning time forwarded. 























Time from mission to order. 





Time to decision ratio. 








Number of orders issued. 








Percent of personnel informed. 





Warning to operations order ratio. 








Ability to develop courses of action. 








Tactical picture quality. 












































Display processing time. 














Table 3. Facilitate Commander's Influence of Events 


The MOEs with the highest performance rankings in Table 3 are "Tactical 


Picture Quality" and "System Contribution to the Commander's Perception of the Enemy." 
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This result is consistent with fundamental concepts of C*I for the Warrior and maneuver 
warfare, which are "to provide the knowledge and understanding the commander needs" 
so that his decision cycle is faster than his opponent's [Ref. 2]. Measures with two of the 
three highest performance rankings in Table 3 also deal with the decision process ("System 
Contribution to Commander's Perception of the Enemy" and "Ability to Develop Courses 
of Action"), while percentage ("Percent Orders Clarification Requested") and event 
measures ("Number of Orders Issued") have lower ranks. Additionally, the two measures 
with the lowest performance rankings, "Display Processing Time" and "Number of Orders 
issued," can be used to measure less than the table average of three supporting elements 
(as measured by the number of X's along a row). Doctrinal requirement (i), dealing with 
providing the commander the ability to view the battlefield beyond current operations, is 
weakly supported with only one MOE (one X in the (i) column). 
2. Doctrinal Category: Adapt to the Situation 
"The commander must be free to move and lead from any point in the battlespace. 

Therefore, the C? system must have the flexibility to adapt to his needs rapidly." [Ref. 24] 

a. Supporting Elements 

(a) A C’ system possess the characteristic of "flexibility." [Ref. 24] 

(b) A C’ system should "communicate those plans and orders (see category 1 
"translate commander's decisions into plans and orders") to subordinates." [Ref. 26] 

(c) A C? system "must be easy to establish, easy to maintain, and reconfigure." 


[Ref. 1] 


4] 


(d) The system "must adapt to rapidly changing situations and exploit fleeting 
opportunities." [Ref. 27] 

(e) Should possess the "capability to support merging forces and expanding to 
larger MAGTFs without disrupting continuity of operations." [Ref. 1] 

(f) Should possess the "capability for the commander to view friendly and 
enemy force data as it is received." [Ref. 1] 

(g) Should possess the "capability to support planning on the move." [Ref. 1] 

(h) "Must satisfy the command and control requirements of the MAGTF 


commander afloat, ashore, or in transition." [Ref. 1] 
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Measure of Effectiveness 


—=— 


Measures These Supporting 


Elements 


Performance 





Time from mission to order. 





Time to decision ratio. 





Set up and tear down time. 








|Percent of communications with alternate routes. 





Grade of service. 





System access. 


= 





Display processing time. 








Database size consistency. 





Tactical picture consistency. 











Mean dissemination time. 





Percent informed. 
| 








'Ability to reconfigure. 





Percent messages displayed inaccurately. 








|— 


Number of orders issued. 





Number of duplicate reports. 








Tactical picture quality. 


















































Table 4. Adapt to the Situation. 


For the measures presented in Table 4, number counts ("Number of Duplicate 


Reports" and "Number of Orders Issued") seem to be ranked lower than time measures 


("Time from Mission to Order" and "Time to Decision Ratio"), which reflects the 


importance of tempo in maneuver warfare, especially when dealing with the commander's 


ability to adapt to a changing situation. Measures dealing with displays and databases also 


seem to be less important than decisionmaking and mobility MOEs. The average number 
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of supporting elements that can be measured by an MOE (the average number of X's ina 
row) is between one and two. 
3. Doctrinal Category: Support Information Requirements 
"The commander's C? support system must provide a reliable, secure, fast and 
durable information network that allows him to implement his operational concepts. . . 
information must be: timely, accurate, complete, objective, useable, relevant, positioned 
properly, mobile, accessible and fused." [Ref. 24] 
a. Supporting Elements 
(a) "There must be a means to provide accurate navigation down to the smallest 
unit to reduce the amount of communications and improve command and control 
effectiveness." [Ref. 1] 
(b) AC? system "provides universal availability of data with selected access 
based on a commander's mission and needs." [Ref. 8] 
(c) AC’ system "provides battle-relevant, theater and national databases having 
common data elements." [Ref. 8] 
(d) A C’ system should "facilitate (the) rapid, distributed and unconstrained 
flow of information in all directions." [Ref. 28] 
(e) AC’ system should "filter, fuze and prioritize information.” [Ref. 28] 
(f) AC’ system should "provide echelon jumping capability." [Ref. 28] 


(g) AC’ system should "provide vertical, lateral, and diagonal redundancy." 


[Ref. 28] 
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(h) AC? system should "provide the commander accurate, timely information 
and ideas for developing feasible courses of action and making logical decisions." [Ref. 
26] 

(i) "Satisfy the commander's critical information requirements, and make those 
elements of information available in a timely fashion and in the most useable form, 


supporting both supply-push and demand-pull." [Ref. 1] 
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Measure of Effectiveness Measures These Supporting Performance 
Elements 
(a) | (b) | (c) | (4) | (@)| (f) | (9) (h) | (i) 

Number of options remaining. xX 14 
Time to decision ratio. | x 9 
Percent commander's critical information Xx Xx xX |X 1 
requirements met. 

|Proportion friendly elements engaged (fratricide). xX 13 
Grade of service. 8 














Ability to develop courses of action. 








Display processing time. 








Database size consistency. 








Tactical picture consistency. 











Digital map capability. 








JAbility to support overlays. 


‘System tasking timeliness. 





Percent of messages displayed inaccurately. 











Average time from threat detection to identification. 











Time potential threat undetected ratio. 








Tactical picture quality. 


Sensor effectiveness. 


7 Table 5. Support Information Requirements. 






























































For Table 5, the measures which gauge the satisfaction of a commander's 
information requirements ("Percent Commander's Critical Information Requirements 
Met"), overall system speed ("System Tasking Timeliness"), and knowledge of the enemy 
("Tactical Picture Quality" and "Average Time from Threat Detection to Identification") 


rank the highest in measuring overall information requirements. Measures of subsystem 
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equipment performance (including "Sensor Effectiveness" and "Display Processing 
Time"), are ranked lowest for measuring support of information requirements. Supporting 
elements (f) and (g), dealing with echelon jumping and system redundancy, are weakly 
supported with only one MOE. 

4. Doctrinal Category: Exploit MAGTF Capabilities 


"The C’ support system must: 


-Support the MAGTF during all types of deployment and employment 

-Enable the MAGTF to maintain mobility (note: this is not the same as commander's 
freedom to move in the battlespace). 

-Survive enemy weapons systems and C? warfare 

-Sustain the MAGTF 

-Provide the flexibility to task organize 

-Interface with other C’ systems (Joint, combined, NCA, other services/federal 
agencies) [Ref. 24] 


a. Supporting Elements 

(a) AC’ system should possess the characteristics of deployability, mobility, 
flexibility, integratability, interoperability [Ref. 24]. 

(b) Should "allow seamless transfer of information within naval forces and 
between naval forces and joint commands." [Ref. 8] 

(c) “Provides rapid simultaneous access of multiple users throughout the chain 
of command and to external commands and agencies as appropriate.” [Ref. 8] 


(d) Communications must be "easy to establish, maintain and reconfigure." [Ref. 


(e) "C? employment options cannot be considered independent of deployment 


options." [Ref. 1] 
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(f) AC? system should have the capability to "deploy rapidly, operate under 
austere conditions and at extended ranges, and project and maintain from a sea-base." 


[Ref. 1] 


(g) AC’ system "should be able to withstand disruptions of all kinds — created 


by the enemy or self-induced." [Ref. 1] 





Measure of Effectiveness Measures These Performance 








Supporting Elements 


















Percent communications links with alternate routes. 











Balance of traffic throughout network. 








|\Satcom supportability. 





Percent informed. 











Ratio of supplies consumed versus provided. 




























































































Mean time response to jamming. 

Personnel requirements improvement. 3 
Weapons systems equivalents. 2 

a | 

Database size consistency. 16 
Tactical picture consistency. 1 
‘Transportation requirements improvement. 8 
Index of availability. 5 
Index of survivability. xX 7 
Transportability index. 4 
System access. 9 
Ability {o reconfigure [ 2 























Table 6. Exploit MAGTF Capabilities 
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For Table 6, those measures which deal with mobility and deployability ("Ability to 
Reconfigure" and "Personnel Requirements Improvement") are ranked high for their 
performance in assessing a C*I system's ability to exploit MAGTF capabilities. 
Interestingly, measures which deal with combat service support ("Ratio of Supplies 
Consumed Versus Provided") and information ("Percent Informed" and "Database Size 
Consistency") are ranked at the bottom. All supporting elements in Table 4 can be 
measured by at least three MOEs. 

5. Doctrinal Category: Support CP Effectiveness 

A command post (C’ system) needs to be able to survive, and also to fight [Ref. 24]. 
A. tradeoff, however, exists between fighting and survival as shown in Figure 6. 


Characteristics that help a C” survive may impede its ability to fight, and vice-versa. 





Survive —_—$ > Fight 

Opsec Tradeoff Speed 

Mobility Simplicity 

Austerity CP Design 

Signature Information 

Location Automation 
Deception Communication 
Dispersion Standardization 
Redundancy Qualified Personnel 
Cover and Concealment Continuous Operation 








Figure 6. Survive and Fight Tradeoff. From Ref. [24]. 
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a. Supporting Elements 

(a) A C? system should possess the characteristics of survivability and 
sustainability. [Ref. 24] 

(b) "Provides rapid simultaneous access of multiple users throughout the chain 
of command and to external commands and agencies as appropriate." [Ref. 8] 

(c) "Displays a scaleable, near real-time, shared picture of the battlefield.” [Ref. 
8] 

(d) "C? facilities must be able to operate on the move." [Ref. 1] 

(e) "There is a need to reduce the number of transmissions and transmission 
time, thereby reducing the electronic signature." [Ref. 1] 

(f) Should facilitate "rapid and frequent displacements." [Ref. 1] 

(g) Organization should "ensure unity of command." [Ref. 27] 

(h) "Organization should also ensure reasonable span of control (3-7 units)."” 
[Ref. 26] 

(i) Must have an "“echelonment capability" (A and B commands for 


displacement). [Ref. 1]. 
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Measures of Effectiveness Measures These Supporting Performance 
Elements 
(a) | (b) (¢) | (d)} (e)| (f) | (9) | (h) | (i) 
ee ai | 
Communications performance index (capacity, X | X X' X|X|]X|X |XX 4 
| 
range, security, mobility, dependability). 
| 
Time to decision ratio, x x x 10 
Average message backlog | 5 














Percent of actions initiated by time ordered. 





Display processing time. 





Percent of net capacity utilization. | =P 19 
J ——P — 
Grade of service. xX X | 13 
x 
| x 


Percent of personnel informed. 





Set-up and tear-down time. 





Ability to reconfigure. 





Personnel requirements improvement. 














System tasking timeliness. 





Percent radio links up. 





Messages lost due to radio links down. 





Number of duplicate reports. 








Tactical picture quality. 











Tactical picture consistency. 








Average message duration. 








Transportability index. 








Index of survivability. 





Index of availability. 












































Table 7. Support CP Effectiveness 


In the category of Supporting CP Effectiveness, MOEs dealing with 


communications ("Communications Performance Index"), system displays and information 
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("Tactical Picture Quality"), and mobility ("Transportability Index") are ranked high in 
performance. Specific counts and percentage measures ("Percent of Personnel Informed" 
and "Number of Duplicate Reports") are assessed as relatively poor measures of CP 


Effectiveness throughout the table. 
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IV. CONCLUSIONS 


A. SUMMARY 

1. Background 

Riding the crest of the "information age" and what has been described as a 
"revolution in military affairs," C*I technologies are emerging as the force multipliers of 
future battlefields. Current initiatives strive to push information and intelligence to 
increasingly lower echelons of command, without exceeding the commander's capacity to 
sift and assimilate what is pertinent. However, due to past shortcomings in C‘I 
procurement, and the high cost of emerging technologies, upgrades to current C’I systems 
and new C‘I acquisitions are subjected to intense analysis. 

2. CI Evaluation Methodologies 

A multitude of C’I evaluation methodologies exist, each approaching the problem of 
comparative analysis in a slightly different manner. Available techniques include the 
COEA, MCES, MOA, HEAT, evolutionary upgrade paths, and functional decomposition 
methods. Importantly, C*I evaluation methodologies can be combined into hybrid 
techniques that benefit from the strengths of individual techniques. 

Current practice in the Marine Corps and other services has established the COEA 
as the de facto framework for conducting C*I system comparative analyses. The COEA 
methodology, however, is broad enough to accept input from other, potentially more 
precise, evaluation techniques. The strength of the MCES lies in its initial focus on 


problem definition and the environment, rather than a premature focus on a quantitative 
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model. MCES also enjoys a commonality of terms and structure which simplify its 
application. The MOA focuses on obtaining required input from political and military 
decisionmakers prior to defining an analytical question. HEAT has been successfully 
applied to military C*I systems, and represents a systematic methodology using 
pre-defined measures. Evolutionary upgrade path concepts help provide greater 
resolution when comparing alternate systems with similar capabilities. Functional 
decompositions use varied approaches which couch an analysis in terms of required 
system capabilities. 

Although any hybrid combination of methodologies would only be developed in the 
context of the analysis at hand, one such combination that suggests itself might be a 
COEA which employs the first four modules of MCES for problem definition, and 
evolutionary upgrade paths for distinguishing the best of a subset of "good" systems. 

3. MOEs 

The identification of realistic MOEs is a common thread which binds the various C*I 
evaluation methodologies. Unique MOEs may be required for a specific analysis, and can 
be generated using MCES, MOA, evolutionary upgrade paths, or functional 
decompositions. Importantly, many useful MOEs already exist from past analyses, C‘I 
literature, and academia. The ACCES, for instance, uses a pre-defined set of measures 
which remain constant from analysis to analysis. The proposed framework for 
categorizing MOEs in Chapter III, based on relevant doctrine, provides a tool for 


identifying and maintaining those MOEs with broad applicability to Marine Corps C’l 
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system evaluation. The framework can change dynamically with emerging doctrine, and 
can also incorporate new MOEs developed by the application of MCES or other MOE 
generating methodologies. 

Importantly, the framework provides a starting point for increased resolution in 
categorizing C*I MOEs for use in Marine Corps related analysis. Potential sub-categories 
that may provide greater detail include functionality (i.e., is a component primarily for C’, 
communications, intelligence, or interoperability?), and compatibility (e.g., is the system 
employable inside existing vehicle spaces and CP configurations?). 

For the measures identified in the literature, those MOEs which go beyond number 
counts and percentages seem to promise better performance in measuring doctrinal C*I 
system requirements. Higher level measures, especially Tactical Picture Quality, enjoyed 
high performance rankings across multiple doctrinal categories. The identification of 
existing measures which provide high performance provides a logical starting point for 


refinement and identification of similarly powerful MOEs. 
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APPENDIX A. MOE DESCRIPTIONS 


A. DEFINITIONS AND RATIONALE 

1. MOE: System Contribution to Commander's Perception of the Enemy 

Definition: The total number of enemy units upon which there is intelligence 
information in a database, the age of the intelligence information, and current time within 
the scenario. The output is a cumulative graph of the number of enemy units upon which 
there is intelligence information versus time. [Ref. 23] 

Rationale: The commander's perception of the battlefield has a direct impact on the 
decision that he will make. This measure attempts to assess the contribution of the C*I 
system to the commander's decision making process, and provide for a comparative 
analysis of alternatives for supporting the commander's decision making process. 

Supports: Facilitate Commander's Influence of Events. 

2. MOE: Percent Action Initiated by Time Ordered 

Definition: The percentage of all actions initiated in response to orders that are 


initiated within the time specified by the order [Ref. 23]. 


Number of actions initiated by time ordered 


Percent Actions Initiated on Time = 


Number of actions ordered 


Rationale: This measure addresses the timeliness of subordinate reaction to orders, 
and by extension the timeliness of the C*I system in processing, transmitting, and 
displaying the orders to subordinate commander's. 


Supports: Facilitate Commander's Influence of Events, Support CP Effectiveness. 
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3. MOE: Mean Dissemination Time 
Definition: The time required to disseminate an order, directive, or warning to all 


elements at the next lower echelon of command [Ref. 23]. 


> (average dissemenation time for each order) 





Mean Dissemenation Time = 


number of orders 


Rationale: This measure directly measures the timeliness of a CI system in 
disseminating orders. 

Supports: Facilitates Commander's Influence of Events, Adapt to the Situation. 

4. MOE: Percent Orders Clarification Requested 

Definition: The percentage of total orders issued (can include frag orders), for 


which any subordinate element requested clarification [Ref. 29]. 


number orders clarification requested 
imunber orders clarification Fequested 4 ae 


Percent Orders Clarification Requested = 


number of orders issued 


Rationale: This measure addresses the clarity of the orders issued, indirectly 
addressing the quality of orders produced by the C’I system. It is also an indirect measure 
of the like-mindedness of subordinate and senior commanders. 

Supports: Facilitates Commander's Influence of Events. 

5. MOE: Time to Decision Ratio 

Definition: The proportion of time from receipt of mission until time of execution 


that is devoted to a commander's decision [Ref. 23]. 
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{time order approved) ~ (time receipt mission) 





Time to Decision Ratio = — 
(time execution ordered) — (time receipt mission) 


Rationale: This measure addresses the effectiveness of a C*l system. It is assumed 
that the C*I system which provides the most relevant information, the most assistance in 
preparing an order, the best standard operating procedures, and the best human interface 
with technology will require less time to finalize an order. 

Supports: Facilitates Commander's Influence of Events, Adapt to the Situation, 
Support Information Requirements, Support CP Effectiveness. 

6. MOE: Percent Planning Time Forwarded 

Definition: The percentage of total planning time available that an echelon allows to 


all lower echelons [Ref. 23]. 


Percent Planning Time Forwarded = (1 - Time to Decision Ratio) X 100 


Rationale: This measure addresses C*I system effectiveness by assessing the 
planning time required to develop and issue an order in relation to the time available. 
Marine Corps doctrine calls for passing fifty percent of the available time to mission to 
subordinates. 


Supports: Facilitate Commander's Influence of Events. 
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7. MOE: Time from Mission to Order 

Definition: The elapsed time at one echelon of command from the moment of 
receiving a mission to the moment of issuing the responsive order to the next lower 
echelon [Ref. 23]. 

Rationale: This measure directly measures the timeliness of the command function. 
It includes planning time, decision time, and time to prepare and disseminate the order. It 
is assumed that the best C*I system will have the fastest time from mission to order. The 
measure may also be used to compare the command function in different environments, 
e.g., mobile vs stationary, or afloat vs ashore. 

Supports: Facilitate Commander's Influence of Events, Adapt to the Situation. 

8. MOE: Number of Orders Issued 

Definition: A simple number count of the orders issued for a given operation [Ref. 
29). 

Rationale: This measure directly measures the amount of command and control and 
is considered an indication of amount needed, which itself relates to cost or burden. 
Additionally, this assesses indirectly the commander's perception of the battlefield (better 
perception relates to less orders), and the extent to which subordinate commanders are 
receiving ideas and concepts (the better they understand the situation the less direction 
they will require). 


Supports: Facilitate Commander's Influence of Events, Adapt to the Situation. 
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9. MOE: Percent of Personnel Informed 


Definition: The percentage of personnel aware of an information item when asked 


[Ref 29]. 


number of personnel aware of item 


x 100 


number of personnel asked 


Rationale: This measure addresses the extent to which a system provides the same 
information to all users, or the extent to which subordinate commanders receive ideas and 
information, or how well a system is performing in different environments (e.g., on the 
move Vs stationary). 

Supports: Facilitate Commander's Influence of Events, Adapt to the Situation, 
Exploit MAGTF Capabilities, Support CP Effectiveness. 

10. MOE: Warning Order to Operations Order Ratio 

Definition: The number of warning orders divided by the number of operations 
orders (can include frag orders) [Ref. 29]. 

Rationale: This measure assesses the extent to which subordinate commanders are 
receiving ideas and concepts, vice data. It also assesses the extent to which all 
commanders are operating off the same tactical picture. 

Supports: Facilitate Commander's Influence of Events. 

11. MOE: Ability to Develop Courses of Action 

Definition: The time necessary to develop courses of action given commander's 


guidance [Ref. 30]. 
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Rationale: This measure assesses the extent to which a C’I system facilitates the 
development of COAs. It is assumed a superior C’I system will be faster than an inferior 
system. 

Supports: Facilitate Commander's Influence of Events, Support Information 
Requirements. 

12. MOE: Tactical Picture Quality 

Definition: Measured by a function of completeness of tactical picture, accuracy of 
identification, and accuracy of position indication [Ref. 31]. 

Rationale: This measure assesses the accuracy of the tactical information provided 
by the C’I system. Tactical picture quality will impact the commander's perception of the 
battlefield, as well as staff planning. 

Supports: Facilitate Commander's Influence of Events, Adapt to the Situation, 
support Information Requirements, Support CP Effectiveness. 

13. MOE: Display Processing Time 

Definition: Duration of time between receipt of information at C*l node and time 
when information first appears at a display where decision-making process begins [Ref. 
31]. 

Rationale: This measure assesses the timeliness of a portion of the C*I system. It is 
assumed that the system with the fastest display processing time will be superior to those 


with slower display processing times. Note that this measure corresponds to the observe 


portion of the Boyd Cycle. 
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Supports: Facilitate Commander's Influence of Events, Adapt to the Situation, 
support Information Requirements, Support CP Effectiveness. 

14. MOE: Set-up and Tear-down Time 

Definition: Set-up time is the elapsed time from CP site selection to operational 
capability of the C*l system. Tear-down time is the time required to dismantle the system 
and load it aboard transportation [Ref. 30]. 

Rationale: This measure addresses the flexibility of a C*I system. A system which 
can be installed and dismantled quickly will be more responsive, survivable, and mobile. 

Supports: Adapt to the Situation, Support CP Effectiveness. 

15. MOE: Percent Communications with Alternate Routes 

Definition: The percentage of all established node to node communications links 
that also have an existing alternate route for communications [Ref. 29]. 

Rationale: This measure addresses the robustness of a C‘I system. Alternate 
communications routes ease message backlogs, resist jamming effects, and increase 
timeliness. 

Supports: Adapt to the Situation, Exploit MAGTF Capabilities. 

16. MOE: Grade of Service 

Definition: The probability that a subscriber at any randomly chosen instant will be 
able to obtain a circuit connection to his party [Ref. 29]. 

Rationale: This measure assesses the responsiveness and quality of a C‘I system. A 


better grade of service suggests a better flow of information between echelons of 
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command and within CPs. Comparing grades of service allows analysis of CPs in different 
states (e.g., mobile vs stationary). 

Supports: Adapt to the Situation, Support Information Requirements, Support CP 
Effectiveness. 

17. MOE: System Access 

Definition: Proportion of time requested that the MAGTF commander was given 
access to a specific (shipboard) system to support (OTH) operations [Ref 31]. 

Rationale: This measure assesses the responsiveness and supportability of a C’l 
system. Due to equipment constraints afloat, as well as satellite access and bandwidth 
constraints afloat or ashore, the MAGTF commander may not enjoy unlimited access to 
C‘lI assets. Examples include limited W-3 shipboard satellite antennas, and limited SHF 
satcom bandwidth for GMF terminals ashore. 

Supports: Adapt to the Situation, Exploit MAGTF Capabilities. 

18. MOE: Database Size Consistency 

Definition: Proportion of database consistency between the MAGTF headquarters 


and selected subordinate units [Ref. 31]. 


Database size consistency = | - (= os ”) 





where: x = database size of MAGTF headquarters 


yn = database size of nth selected subordinate unit 
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Rationale: This measure addresses the consistency of a C*I system, and the extent 
to which commanders and staffs at different echelons have access to the same amount of 
information. 

Supports: Adapt to the Situation, Support Information Requirements, Exploit 
MAGTF Capabilities. 

19. MOE: Tactical Picture Consistency 

Definition: Measured by relative comparisons of displays, similarity of 
identification, and similarity of position information [Ref. 31]. 

Rationale. This measure assesses the extent to which commanders and staffs at 
different echelons are operating with the same tactical picture. It may be correlated with 
number of orders issued, database size consistency and percent orders correction 
requested. 

Supports: Adapt to the Situation, Support Information Requirements, Exploit 
MAGTF Capabilities, Support CP Effectiveness. 

20. MOE: Ability to Reconfigure 

Definition: Measured by time to initialize system, time to backup system, and time 
to reboot system [Ref. 30]. 

Rationale: This system addresses C’I system flexibility and mobility. It is assumed 
that a quicker time for each element of the measure is desirable. 

Supports: Adapt to the Situation, Exploit MAGTF Capabilities, Support CP 


Effectiveness. 
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21. MOE: Percent Messages Displayed Inaccurately 

Definition: The percentage of messages received that were displayed inaccurately 
[Ref. 29]. 

Rationale: This measure assesses C*I system accuracy and system information 
reliability. 

Supports: Adapt to the Situation, Support Information Requirements. 

22. MOE: Number of Duplicate Reports 

Definition. A number count of the number of individual reports displayed multiple 
times [Ref. 31]. 

Rationale: This measure addresses the quality of a C*I system. It also forms a basis 
for comparison between CPs in different states (e.g., does the mobile CP have more 
duplicate reports than the stationary CP. 

Supports: Adapt to the Situation, Support CP Effectiveness. 

23. MOE: Number of Options Remaining 

Definition: The number of decision points open multiplied by the number of options 
for each decision point [Ref. 23]. 

Rationale: This measure addresses the quality of a system in supporting planning for 
courses of action, and the amount of flexibility left to the commander. The C*I system 
which presents the greater number of tactical options for a commander is assumed to be 

superior. 


Supports: Support Information Requirements. 
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24. MOE: Percent Commander's Critical Information Requirements Met 

Definition: The percentage of commander's critical information requirements met, 
where commander's critical information requirements are defined by Marine Corps 
doctrine [Ref. 30]. 

Rationale: This measure assesses the quality of a system. The C*I system which 
presents the greatest percentage of CCIRs is assumed superior. 

Supports: Support Information Requirements. 

25. MOE: Proportion Friendly Elements Engaged (Fratricide) 

Definition: The number of friendly elements erroneously engaged divided by the 
number of friendly elements in the engagement area [Ref. 23]. 

Rationale: This measure addresses one of the most catastrophic failures in C’I, 
"friendly fire." By extension, this measure assesses the quality of the tactical picture at 
lower echelons, and the extent to which all echelons are operating from the same tactical 
picture. 

Supports: Support Information Requirements. 

26. MOE: Digital Map Capability 

Definition: Measured by time to generate map display and time to generate scanned 
map [Ref,. 30]. 

Rationale: This measure addresses the timeliness of the C*l system with regard to 


map displays. 


Supports: Support Information Requirements. 





27. MOE: Ability to Support Overlays 

Definition: Measured by time to generate map overlays and time to disseminate map 
overlays to lower echelons other staff sections [Ref. 30]. 

Rationale: This measure addresses the timeliness of a C*I system with regard to 
overlays. 

Supports: Support Information Requirements. 

28. MOE: System Tasking Timeliness 

Definition: The time information is received from a database inquiry minus the time 
the inquiry was initiated [Ref. 31]. 

Rationale: This measure assesses the timeliness of a C*I system. It can also be used 
to compare the system timeliness in different configurations (e.g., mobile or stationary). 

Supports: Support Information Requirements, Support CP Effectiveness. 

29. MOE: Average Time from Threat Detection to Identification 

Definition: The average of all times from threat detection to threat identification 
[Ref. 32]. 

Rationale: This measure addresses the timeliness of the intelligence function of a 
C’'l system. 


Supports: Support Information Requirements. 
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30. MOE: Time Potential Threat Undetected Ratio 

Definition: The length of time an enemy threat is within the system sensor envelope 
before detection divided by the total length of time an enemy threat is within the envelope 
[Ref. 32]. 

Rationale: This measure addresses the sensor effectiveness of a C’l system. By 
extension, the measure can be used to assess the timeliness of reporting within the system 
(i.e., time before detection can actually be time before threat detection report is displayed 
at a particular echelon). 

Supports: Support Information Requirements. 

31. MOE: Sensor Effectiveness 

Definition: The number of reports received by sensor type divided by the total 
number of sensor reports received [Ref. 31]. 

Rationale: This measure assesses the effectiveness of the sensors associated with a 
C’l system. 

Supports: Support Information Requirements. 

32. MOE: Balance of Traffic Through Network 

Definition: The average utilization of each communications node and link [Ref. 29]. 

Rationale: This measure assesses the criticality of communications nodes and links, 
as well as the overall system survivability and compactness of the system (e.g., a system 
with low utilization of many links may in fact have extraneous parts). 


Supports: Exploit MAGTF Capabilities. 
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33. MOE: SatCom Supportability 


Definition: The theoretical satellite uplink channel capacity is measured by: 


R=WS/ ® ++ 9) 2) 
where: 
R = Transmitted data rate capacity of a satellite uplink channel 
W = spread spectrum bandwidth 
S = uplink signal Effective Instantaneous Radiated Power (EIRP) 
E/N, = required post correlation signal-to-noise 
y = signal suppression to the transponder 
J = enemy jammer EIRP 
I = sum of other uplink signals' EIRP 
E = satellite full output downlink EIRP 
L = downlink path loss 
«x = Boltzman’s constant 
G/T = receiver Earth terminal figure of merit 
[Ref. 29] 
Rationale: This measure assesses the amount of bandwidth available to a C’I system 
utilizing SHF satcom communications. 


Supports: Exploit MAGTF Capabilities. 


70 


34. MOE: Ratio of Supplies Consumed Versus Provided 

Definition: The ratio of the quantity of supplies (by class) consumed to the quantity 
of supplies provided by the CSS system [Ref. 23]. 

Rationale: This measure addresses the relative degree to which supply consumption 
can be satisfied by the CSS system. It is assumed that the faster C*I system will speed and 
enhance the satisfaction of supply demands thereby reducing the ratio toward one. 

Supports: Exploit MAGTF Capabilities. 

35. MOE: Mean Time Response to Jamming 

Definition: The arithmetic mean of each elapsed response time to jamming of 
friendly communications [Ref. 33]. 

Rationale: This measure addresses the effectiveness of one countermeasure aspect 
of a C’l system. 

Supports: Exploit MAGTF Capabilities. 

36. MOE: Personnel Requirements Improvement 

Definition: The net change in personnel requirements, including operators and 
maintainers, of one C*I system over another [Ref. 34]. 

Rationale: This measure assesses the personnel overhead and, by extension, the 
complexity of a C’I system. Results can also be linked to system deployability. 


Supports: Exploit MAGTF Capabilities, Support CP Effectiveness. 
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37. MOE: Weapons Systems Equivalents 

Definition: The number count of a particular weapons system which can be 
considered equivalent to the C*I system in question (e.g., 1 JTIDS equipped F-15 is 
equivalent to 1.3 non-JTIDS equipped F-15s) Ref 35]. 

Rationale: This measure assesses the force effectiveness of a C*l system, and puts 
the result in terms easily understandable to multiple warfighting specialties (e.g., an armor 
officer can readily visualize the value of a CI system if he knows its tank battalion 
equivalence). The enumeration of this measure will usually require a separate analysis. 

Supports: Exploit MAGTF Capabilities. 

38. MOE: Transportation Requirements Improvement 

Definition: The net change in transportation requirements of one C*I system over 
another [Ref. 34]. 

Rationale: This measure addresses C*I system deployability and support 
requirements. It may also provide a measure of tactical mobility. 

Supports: Exploit MAGTF Capabilities. 


39. MOE: Index of Availability 


Definition: Consists of some or all of the following availability measures: 


Operational Availability = (MTBF) / (MTBF + MDT) 


Inherent Availability = (MTBF) / (MTBF + MTTR) 


Achieved Availability = (MTBF) / (MTBF + MTTR + MTTP) 
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where: 
MTBF = mean time between failure 
MDT = mean down time 
MTTR = mean time to repair 
MTTP = mean time to perform scheduled maintenance 
(Ref. 32]. 
Rationale: This measure addresses reliability and availability aspects of a C*I 
system. 
Supports: Exploit MAGTF Capabilities, Support CP Effectiveness. 
40. MOE: Index of Survivability 
Definition: Equals: 


(1-Grade of Service ) / Grade of Service, 


during jamming before jamming 


[Ref. 32]. 

Rationale: This measure assesses system survivability in terms of communications 
disruption. 

Supports: Exploit MAGTF Capabilities, Support CP Effectiveness. 

41. MOE: Transportability Index 


Definition: 


m,)(d,/da)  ifd,24d, 
ifd,,< d, 





where: 
T, = Transportability index 
m, = total jth subsystem volume 
d,, = average density of jth subsystem 
d, = maximum load density for a fully loaded vehicle 
[Ref. 32]. 
Rationale: This measure assesses deployability, mobility, and logistics requirements 
of a C’l system. 
Supports: Exploit MAGTF Capabilities, Support CP Effectiveness. 
42. MOE: Communications Performance Index 


Definition: The weighted sum of the ratios of observed performance to required 


performance over a set of requirements [Ref. 29]. 


CPI = W, (P,/R,) + W, (P,/R,) + ... + W, (P,/R,) 
where: 

W, = relative weight of each requirement 

P, = performance observed 


R, = required performance 


Rationale: This measure is a generic assessment of a CI system over a number of 
xahonale: 8g y: 


stated requirements. Requirements could include: direct communication capacity, organic 
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communication equipment, conference call capability, specific range, security, mobility, 
dependability, vulnerability, etc. 

Supports: Support CP Effectiveness. 

43. MOE: Average Message Backlog 

Definition: The average number of messages awaiting transmission [Ref. 29]. 

Rationale: This measure addresses the timeliness of a C*I system. 

Supports: Support CP Effectiveness. 

44, MOE: Percentage of Net Capacity Utilization 

Definition: The total time a communication net carries traffic divided by the total 
time a net is observed [Ref. 29]. 

Rationale: This measure can be used to address the necessity of a circuit, but is 
more often used to determine whether circuits approach overloading. 

Supports: Support CP Effectiveness. 

45. MOE: Percent Radio Links Up 

Definition: The number of radio links operational divided by the total number of 
radio links [Ref. 35]. 

Rationale: This measure addresses the performance of the communications aspects 
of a C’I system, and can be used to compare system performance in different locations and 
States. 


Supports: Support CP Effectiveness. 
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46. MOE: Number of Messages Lost Due to Radio Links Down 

Definition: A number count of the messages that were not transmitted or which lost 
their time-related value as a result of a radio circuit being down [Ref. 35]. 

Rationale: This measure addresses the reliability of information flow in a C‘l 
system. 

Supports: Support CP Effectiveness. 

47. MOE: Average Message Duration 

Definition: The sum of message transmission times divided by the number of 
messages [Ref. 29]. 

Rationale: This measures addresses the survivability of a C*I system in an EW 
environment. 


Supports: Support CP Effectiveness. 
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APPENDIX B. ANALYSIS OF QUESTIONNAIRE DATA 


Questionnaires were completed by twelve respondents, including eleven Marine 
Officers and one professor. The Marine participants included two infantry officers (both 
company grade), two artillery officers (one field grade, one company grade), and six 
communications officers (one field grade and five company grade), who came from either 
the Joint C’ , the Electrical Engineering, or the Operations Research curriculums. The 
faculty participant was an Operations Research professor with extensive expertise in 
combat modelling. The questionnaires are enclosed at the end of this appendix. 

Respondents were initially asked to rank the MOEs based on three categories, 
including cost, ease of use, and performance. In practice, however, the questionnaires 
proved to be an unsuitable method to obtain cost and ease of use data, since both 
categories are most often a function of the C*I system being analyzed. Performance 
proved to be a tractable category, and complete ranked lists were obtained from each of 
the 12 participants. 

The ranked lists were analyzed using a variant of the Method of Successive Interval 
Scaling, where experts are asked to rank instances in terms of how much of a property 
they possess [Ref 25]. If X, represents an expert's interval scaled feelings about the 
amount of the property possessed by instance j, then the methodology requires two 
assumptions. The assumptions are: 

(1) over an infinite number of experts, X; is distributed normally with mean s, and 


homogeneous variance 0”, for all j. 
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(2) The correlation coefficient between X, and X; is the same for all i,j. 
In applying the method, a frequency matrix of f, values was obtained by counting the 
number of times the MOE in column j was ranked above the MOE in row i. Table 8 


displays the f, matrix resulting from the first doctrinal bin. 

















































































































Table 8. Frequency Matrix 


Next, all the values in the f, matrix are converted to proportions by dividing each f, 


value by f, + fj 


f» aS Shown in Table 9. Note that the diagonal entries are generated on the 
premise that for judges who did not know they were ranking an instance against itself, one 


would expect half the judges to go one way, and half the other way. . 
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1 0.91 | 082 | 045 0.82 
[L aise | — 


Table 9. Proportion Matrix 








Using the normality assumption, each of the proportions may be mapped into a 
standard normal distribution by using the inverse cumulative distribution function. By 
convention, proportions greater than .98 or less than .02 are treated as gaps in the matrix. 


Table 10 displays the resulting z, matrix. 
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set a esl | sai 
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Table 10. Z,, Matrix 


For those columns without holes a scale value for the MOE listed in the column 
header can be generated by the column mean. In Table 10, columns B,E,F,I and J are 


complete. The remaining columns may be solved as a set of simultaneous equations of the 


form, ns,-~ Ls; = zy j=1,2,..m. The solutions to this set of equations will be a 
scale value for the MOE, estimating the column mean. For the problem at hand, the 
following set of equations is generated: 

10S, - S.- S, - Sg- 8, - 8, = 6.7489 

118,,-S,.~ 8-5-5, - 6, - $= 077065 

108..- 8-8, «S,>S, > 8,5 56989 

108-8, - S,-S,- 8, - 5, 2.99781 


10S,, - S, - S, - Sy - Sy = Sy, =-9.19499 
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11S, -S, -S.-S,-S,-S,-S, = 4.19441 

OS. -'§,+8.- 8, <8, =687091 

9S,,- So - S,- Sg - Sy = -5.72169 

where S, equals the scale value for column i. In this instance, the resulting interval 
scale values all fall between -1 and 1, and may be linearly transformed into a more 
appropriate scale. In this case, the linear transformation Y = SOX + 50 was used to put 
the values on a scale between 0 and 100. Scaling results are shown graphically in Figure 


Te. 
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MOE Rank 








Figure 7. Scale Values for MOEs in First Doctrinal Bin 





Figures 8, 9, 10, and 11 graphically display the scaling results for the final four pages 


of the questionnaire. 
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Figure 9. Scale Values for MOEs in Third Doctrinal Bin 
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Figure 11. Scale Values for MOEs in Fifth Doctrinal Bin 
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A. Time from missiou to order. 





B. Time to decision ratio: 





= ( lime issue order to echelon = time receipt mission ) 


time ordered lo start execution — time receipt mission 











D. Percent communications with alternate routes. 


C. Set-up and tear-dowa time. [ — 





E. Grade of service: Probability that at any randomly chosen instant a subscnber is able to obtain a circuit connection to 
his party. 


+ 





F, System Access: Proportion of time requested that the MAGTF commander is given access to a specific shipboard 
system in support of OTH operations 








G. Display processing time: equals (time of display) - (time of receipt) 





H. Database size consistency: is everyone operating from the same (size) database? 





L Tactical picture consistency: measured by relative comparison of pictures, similarity of identification, similarity of 
posiuon information 





J. Mean dissemination time; equals the sum of (the average disseminauon time for each order)/(# of orders) 





K. Percent informed: equals (# of personnel aware of item)/(# of personnel asked) X 100 





L. Ability to reconfigure: time to initialize system, backup system, reboot system. 











M. Percent messages displayed inaccurately. 





N. Number of orders issued. $ 





O. Number of duplicate reports. 





P. Tactical picture quality: measured by completeness of tactical picture, accuracy of identification information, and 
accuracy of position information. 




















Rank lowest to highest, easiest to hardest, and best to worst 








Cost is defined as the monetary expense (including personnel costs) required 
to generate values for the measures 


Ease of Use is defined as the computational complexity involved in assigning 
values to the measure, given data. 


cpmmander’s plans to subordinates. 





Performance 1s defined as the extent to which the MOE provides a realistic 
assessment of how well a C’ system Adapts to the situation. 





ADAPT TO THE SITUATION: ‘The communder must be free to move und lead from uny 
point in the battlespace, Therefore, the C2 system must have the flexibility to adapt to his 
needs rapidly. The C2 system should be easy to establish, maintain and reconfigure, and 
will possess the capability to support merging forces, and expanding to larger MAGTFs, 
without disrupting the continuity of operations. The system should support planning on 
the move, and must satisfy the command and contro] requirements of the MAGTF 
commander afloat, ashore, or in transition. The C2 system must communicate the | 
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A. Percent communications links with alternate routes. 





B. Balance of traffic through network: average utilization of each node and link 





C. SatCom supportability: measured by theoretical uplink channel cpacity, can include yammung degradation. 





D. Percent informed: equals (# of personnel aware of item)/(# of personnel asked) X 100 





E. Ratio of supplies consumed vs supplies provided. 





F. Mean time response to jamming. 





G. Personnel requirements improvement: change in total personnel requirements over another system 





H. Weapons systems equivalents: can the system be equated to a weapons system? (e.g. | JTODS equipped F-15 = 1.3 
non-JTIDS equipped F-1 5s) 


L Database size consistency: is everyone operating from the same (size) database? 








J. Tactical picture consistency: measured by relative comparison of pictures, similarity of identification, similarity of 


posiuon information 


K. Transportation requirements improvement. 














L. Index of availability: consists of operational availability, inherent availability, and achieved availability, all of which | 
are computed using combinations of MTBF, MTTR, MDT, MTTP. 





M. Index of survivability: (1-Grade of Service during jamming)/(Grade of Service before jamming) 








N. Transportability index: computed using subsystem physical size (volume), subsystem density (quantity), and max 
load for fully loaded vehicles, 








O, System Access: Proportion of time requested that the MAGTF commander is given access to a specific shipboard 
system in support of OTH operations. 

















P. Ability to reconfigure: time to initialize system, backup system, reboot system 














Rank lowest to highest, easiest to hardest, and best to worst | ep orr MAGTF CAPABILITIES: The C2 system must: support the MAGTF dunng all 
: types of deployment and employment, enable the MAGTF to maintain mobility, survive 
ss re et = te neues personnel eosis) enemy weapons systems and C2 warfare, sustain the MAGTF, provide the flexibility to task 

: organize, and interface with other C2 systems (joint, combined, NCA, other services/federal 
agencies), C2 employment options cannot be considered independent of deployment options 
The system should provide simultaneous access to muluple users throughout the command, and 
allow for seamless transfer of information within naval forces and between naval forces and 
joint commands, - 


Ease of Use is defined as the computational complexity involved in 
assigning values to the measure, given dala. 


Performance is defined as the extent to which the MOE provides a 
realisuc assessment of how well a C’ system exploits MAGTF capabilities. 
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